Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

lightning: fix encode kvs size greater than 4.0g caused pebble panic #1105

Merged
merged 7 commits into from
May 20, 2021
Merged

lightning: fix encode kvs size greater than 4.0g caused pebble panic #1105

merged 7 commits into from
May 20, 2021

Conversation

3pointer
Copy link
Collaborator

What problem does this PR solve?

panic: pebble: batch too large: >= 4.0 G

What is changed and how it works?

sends kv batch every 96k.

Check List

Tests

  • Unit test
  • Integration test

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Note

  • Fix the issue that lightning panic due to batch kv greater than 4.0g.

@ti-chi-bot ti-chi-bot added size/M and removed size/S labels May 14, 2021
@3pointer 3pointer marked this pull request as ready for review May 14, 2021 09:24
@3pointer 3pointer requested review from kennytm, glorv and Little-Wallace and removed request for YuJuncen May 14, 2021 09:25
@3pointer @glorv
Update pkg/lightning/restore/restore.go
023d2a2 
Co-authored-by: glorv <glorvs@163.com>
Little-Wallace
Little-Wallace approved these changes May 18, 2021
View reviewed changes
Copy link
Collaborator

@Little-Wallace Little-Wallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 LGTM1 label May 18, 2021
@3pointer
Copy link
Collaborator Author

/run-all-tests

@3pointer
Copy link
Collaborator Author

/rebuild

@glorv
Copy link
Collaborator

glorv commented May 20, 2021

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Little-Wallace
  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels May 20, 2021
@glorv
Copy link
Collaborator

glorv commented May 20, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 846569b

@3pointer
Copy link
Collaborator Author

/run-integration-tests

@glorv
Copy link
Collaborator

glorv commented May 20, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 75b8c03

@ti-chi-bot ti-chi-bot merged commit d1aa0ac into pingcap:master May 20, 2021
@ti-chi-bot ti-chi-bot mentioned this pull request May 20, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1127.

@ti-chi-bot ti-chi-bot mentioned this pull request May 20, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1128.

ti-chi-bot added a commit that referenced this pull request May 26, 2021
@ti-chi-bot
lightning: fix encode kvs size greater than 4.0g caused pebble panic (#…
0a47650 
…1105) (#1127)
ti-chi-bot added a commit that referenced this pull request May 27, 2021
@ti-chi-bot
lightning: fix encode kvs size greater than 4.0g caused pebble panic (#…
506d821 
…1105) (#1128)
3pointer added a commit to 3pointer/br that referenced this pull request Jun 4, 2021
@3pointer
lightning: fix encode kvs size greater than 4.0g caused pebble panic (p…
022eb13 
…ingcap#1105)
3pointer added a commit to 3pointer/br that referenced this pull request Jun 4, 2021
@3pointer
lightning: fix encode kvs size greater than 4.0g caused pebble panic (p…
451b3c7 
…ingcap#1105)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sleepymole sleepymole sleepymole left review comments

@glorv glorv glorv left review comments

@Little-Wallace Little-Wallace Little-Wallace approved these changes

@overvenus overvenus Awaiting requested review from overvenus

@kennytm kennytm Awaiting requested review from kennytm

Assignees
No one assigned
Labels
component/import needs-cherry-pick-release-4.0 needs-cherry-pick-release-5.0 size/M status/can-merge status/LGT2 LGTM2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@3pointer @glorv @ti-chi-bot @Little-Wallace @sleepymole